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Abstract 


This paper reviews the corpus of research on feedback, with a particular focus on formative 
feedback—defined as infonnation communicated to the learner that is intended to modify the 
learner’s thinking or behavior for the purpose of improving learning. According to researchers in 
the area, formative feedback should be multidimensional, nonevaluative, supportive, timely, 
specific, credible, infrequent, and genuine (e.g., Brophy, 1981; Schwartz & White, 2000). 
Formative feedback is usually presented as infonnation to a learner in response to some action 
on the learner’s part. It comes in a variety of types (e.g., verification of response accuracy, 
explanation of the correct answer, hints, worked examples) and can be administered at various 
times during the learning process (e.g., immediately following an answer, after some period of 
time has elapsed). Finally, there are a number of variables that have been shown to interact with 
formative feedback’s success at promoting learning (e.g., individual characteristics of the learner 
and aspects of the task). All of these issues will be discussed in this paper. This review concludes 
with a set of guidelines for generating formative feedback. 

Key words: Assessment, formative feedback, directive feedback, facilitative feedback, learning, 
performance, individual differences, goal orientation, motivation, task characteristics 
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1. Introduction 


It is not the horse that draws the cart, but the oats. -Russian proverb 

Feedback used in educational contexts is generally regarded as crucial to improving 
knowledge and skill acquisition (e.g., Azevedo & Bernard, 1995; Bangert-Drowns, Kulik, Kulik, 
& Morgan, 1991; Corbett & Anderson, 1989; Epstein et ak, 2002; Moreno, 2004; Pridemore & 
Klein, 1995). In addition to its influence on achievement, feedback is also depicted as a 
significant factor in motivating learning (e.g., Lepper & Chabay, 1985; Narciss & Huth, 2004). 
However, for learning, the story on feedback is not quite so rosy or simple. 

According to Cohen (1985), feedback “... is one of the more instructionally powerful and 
least understood features in instructional design” (p. 33). In support of this claim, consider the 
hundreds of research studies published on the topic of feedback and its relation to learning and 
performance during the past 50 years (for excellent historical reviews, see Bangert-Drowns et ah, 
1991; Kluger & DeNisi, 1996; Kulhavy & Stock, 1989; Kulhavy & Wager, 1993; Mory, 2004; 
and Narciss & Huth, 2004). Within this large body of feedback research, there are many 
conflicting findings and no consistent pattern of results. 

Consider just one facet of feedback: elaboration (i.e., explanatory infonnation within a 
feedback message). Some studies report that elaborative feedback produces significantly greater 
learning among students compared with feedback containing less information (e.g., Albertson, 
1986; Grant, McAvoy, & Keenan, 1982; Hannafin, 1983; Moreno, 2004; Pridemore & Klein; 
1995; Roper, 1977; Shute, 2006). However, other studies show that increasing the amount of 
feedback information has no effect on learning or perfonnance (e.g., Corbett & Anderson, 1989, 
1990; Gilman, 1969; Hodes, 1985; Kulhavy, White, Topp, Chan, & Adams, 1985; Merrill, 

1987). The goal of this paper is to try to make sense of the tangled web of formative feedback 
research and its relationship to learning. I start by defining relevant tenns. 

1.1 Definition of Formative Feedback 

Formative feedback represents infonnation communicated to the learner that is intended 
to modify the learner’s thinking or behavior for the purpose of improving learning. And while 
the teacher may also receive formative feedback and use it as the basis for altering instruction, I 
focus on the student (or more generally, the learner) as the primary recipient of formative 
feedback in this review. 
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Information within the feedback may address the accuracy of a response to a problem or 
task and may additionally touch on particular errors and misconceptions (Azevedo & Bernard, 
1995; Birenbaum & Tatsuoka, 1987; Cheng, Lin, Chen, & Heh, 2005; Cohen, 1985; Kulhavy, 
1977; Sales, 1993; Sleeman, Kelly, Martinak, Ward, & Moore, 1989), the latter representing 
more specific or elaborated types of feedback. To be effective, formative feedback should permit 
the comparison of actual perfonnance with some established standard of perfonnance (Johnson 
& Johnson, 1993). The definition of formative feedback may be further refined as 
multidimensional, nonevaluative, supportive, learner-controlled, timely, specific, credible, 
infrequent, contingent, and genuine (e.g., Brophy, 1981; Schwartz & White, 2000). 

In technology-assisted instruction, similar to classroom settings, formative feedback 
comprises information—a message, display, and so on—presented to the learner following the 
learner’s input (or upon request, if applicable), with the purpose of shaping the perception, 
cognition, or action of the learner (e.g., Moreno, 2004; Schimmel, 1983; Wager & Wager, 1985). 
The main goal of formative feedback—whether delivered by a teacher or computer, in the 
classroom or elsewhere—is to enhance learning and/or perfonnance, engendering the formation 
of accurate, targeted conceptualizations and skills. Such feedback may be used in conjunction 
with low- or medium-stakes assessments, include diagnostic components, and even be 
personalized for the learner (Albertson, 1986; Azevedo & Bernard, 1995; Narciss & Huth, 2004; 
VanLehn, 1982). 

The premise underlying most of the research conducted in this area is that good feedback 
can significantly improve learning processes and outcomes, if delivered correctly. Those last 
three words—if delivered correctly—comprise the crux of this review. This paper reviews the 
literature on feedback, with the goal of cataloging and categorizing key facets (i.e., types and 
timing), in addition to other variables with which they are known to interact (e.g., learner states 
and traits and task complexity). 

1.2 Goals of the Review 

The dual aims of this paper are to (a) present findings from an extensive literature review 
of feedback to gain a better understanding of the features, functions, interactions, and links to 
learning and (b) apply the findings from the literature review to create a set of guidelines relating 
to fonnative feedback. The overarching goal is to identify the particular features of formative 
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feedback that are the most effective and efficient in promoting learning and determine under 
what conditions that learning support holds. 

This is not an easy task. The vast literature reveals dozens of feedback types that have 
been subjected to experimental scrutiny—accuracy of the solution, topic contingent, response 
contingent, attribute isolation, worked examples, hints, and partial solutions. However, different 
studies report disparate findings regarding the same feedback variable. In addition, fonnative 
feedback variables have been shown to interact with other variables, such as student achievement 
level, task level, and prior knowledge. Again, my goal is to capture and integrate these feedback 
features so to begin to define prescriptions relating to formative feedback and its appropriate 
delivery. 

1.3 Focus of the Review 

This review focuses on task-level feedback as opposed to general summary feedback. 
Task-level feedback typically provides more specific and timely (often real-time) infonnation to 
the student about a particular response to a problem or task compared to summary feedback, and 
it may additionally take into account the student’s current understanding and ability level. For 
instance, a struggling student may require greater support and structure from a formative 
feedback message than a proficient student. On the other hand, summary information is useful 
for teachers to modify instruction for the whole class and for students to see how they are 
generally progressing. 

Some of the major questions addressed in this review include: What are the most 
powerful and efficient types of formative feedback and under what conditions do these different 
types of feedback help a learner revise a skill or improve understanding? What are the 
mechanisms by which feedback facilitates the transformation of rudimentary skills into the 
competence of a more expert state? Answers to these questions can facilitate the design and 
development of teacher-delivered or automated feedback to support learning. 

This paper begins with a summary of the methods used to accomplish the literature 
review, followed by an extensive review of formative feedback research, which comprises the 
bulk of the paper. Afterwards, I showcase four important feedback articles, each associated with 
a theoretically and empirically based model of formative feedback. I conclude with specific 
recommendations for using fonnative feedback that are supported by the current literature review 
and discuss future research in the area. 
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2. Method 


2.1 Procedure 

I began my research by working with colleagues to identify seminal articles in the 
feedback literature (i.e., from sites that provide indices of importance such as CiteSeer) and then 
collecting the relevant documents. The bibliography compiled from this initial set of research 
studies spawned a new collection-review cycle, garnering even more articles and continuing 
iteratively throughout the review process. 

The following online databases were employed in our search-collection efforts: 

• ERIC (Educational Resources Infonnation Center) contains infonnation on 
educational reports, evaluations, and research, including data from Resources in 
Education Index and Current Index to Journals in Education. 

• PsycINFO, from the American Psychological Association, includes citations and 
summaries of scholarly journal articles, book chapters, books, and dissertations in 
psychology and related disciplines. 

• PsycARTICLES is a source of full-text, peer-reviewed scholarly and scientific articles 
in psychology. The database covers general psychology and specialized, basic, 
applied, clinical, and theoretical research. It contains articles from 56 journals (45 
published by the American Psychological Association and 11 from allied 
organizations). 

• Academic Search Premier offers information in many areas of academic study 
including computer science, engineering, physics, language and linguistics, and so 
forth. 

• MasterFILE Premier, designed specifically for public libraries, covers a broad range 
of disciplines including general reference, business, education, health, general 
science, and multicultural issues. 

In addition to these databases, other research sources included the online catalogs at the 
libraries at ETS and the University of Pennsylvania, which were used to access their electronic 
collections of journals and research studies. Another source was Google Scholar—a Web site 
providing peer-reviewed papers, theses, books, abstracts, and articles from academic publishers, 
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professional societies, preprint repositories, universities, and other scholarly organizations, to 
search for and acquire specific references. 

2.2 Inclusion Criteria 

The review focused on full-text documents, accessed using various search terms or 
keywords such as feedback, formative feedback, formative assessment, instruction, learning, 
computer-assisted/based, tutor, learning, and performance. Searches were not limited to a 
particular date range, but did include the most recent research in the literature review. In all, 
approximately 170-180 articles, dissertations, abstracts, books, and conference proceedings were 
collected. From this larger set, a total of more thanlOO documents met the study’s criteria for 
inclusion into the literature review. Inclusion criteria consisted of topical relevance, use of 
experimental design, and/or meta-analytic procedures (albeit, this selection process was more 
akin to that of community colleges than ivy-league universities). The majority of the documents 
were journal articles (103), followed by books and book chapters (24), conference proceedings 
(10), and other (e.g., research reports, 4). 

3. Literature Review 

As noted at the beginning of this paper, hundreds of articles have been written about 
feedback and its role in knowledge and skill acquisition. Many of these papers describe the 
results from experimental tests examining different features of feedback, and several represent 
important historical reviews (a few going back to the early 1900s, such as Kluger & DeNisi, 
1996; Kulhavy & Stock, 1989; and Mory, 2004). Despite the plethora of research on the topic, 
the specific mechanisms relating feedback to learning are still mostly murky, with very few (if 
any) general conclusions. Researchers who have tackled the tough task of performing meta¬ 
analyses on the feedback data use descriptors such as inconsistent, contradictory, and highly 
variable to describe the body of feedback findings (Azevedo & Bernard, 1995; Kluger & 

DeNisi). Ten years later those descriptors still apply. 

Feedback has been widely cited as an important facilitator of learning and performance 
(Bandura, 1991; Bandura & Cervone, 1983; Fedor, 1991; Ilgen, Fisher, & Taylor, 1979), but 
quite a few studies have reported feedback as having either no effect or debilitating effects on 
learning (for examples of nonfacilitative effects of feedback on learning, see Bangert-Drowns et 
ah, 1991; Kluger & DeNisi, 1996; and Mory, 2004). In fact, about one third of the total studies 
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reviewed in two landmark meta-analyses (i.e., Bangert-Drowns et al.; Kluger & DeNisi) 
demonstrate negative effects of feedback on learning. For instance, feedback that is construed as 
critical or controlling (Baron, 1993) often thwarts efforts to improve performance (Fedor, Davis, 
Maslyn, & Mathieson, 2001). Other features of feedback that tend to impede learning include 
providing grades or overall scores indicating the student’s standing relative to peers and coupling 
such nonnative feedback with low levels of specificity (i.e., vagueness) (Butler, 1987; Kluger & 
DeNisi, 1998; McColskey & Leary, 1985; Wiliam, in press; Williams, 1997). In addition, 
interrupting a student who is actively engaged in problem solving with feedback from an external 
source has too has been shown to inhibit learning (Corno & Snow, 1986). In line with the 
definition in this review, feedback that has negative effects on learning is not fonnative. 

3.1 Feedback Purposes 

The main aim of formative feedback is to increase student knowledge, skills, and 
understanding in some content area or general skill (e.g., problem solving), and there are 
multiple types of feedback that may be employed toward this end (e.g., response specific, goal 
directed, immediately delivered, and so on). In addition to various formats of feedback, there are 
also different functions. According to Black and Wiliam (1998), there are two main functions of 
feedback: directive and facilitative. Directive feedback tells the student what needs to be fixed or 
revised. Such feedback tends to be more specific than facilitative feedback, which provides 
comments and suggestions to help guide students in their own revision and conceptualization. 

The next section describes some of the ways that feedback may exert influences on student 
learning. 

3.2 Cognitive Mechanisms and Formative Feedback 

There are several cognitive mechanisms by which formative feedback may be used by a 
learner. First, it can signal a gap between a current level of performance and some desired level 
of performance or goal. Resolving this gap can motivate higher levels of effort (Locke & 

Latham, 1990; Song & Keller, 2001). That is, formative feedback can reduce uncertainty about 
how well (or poorly) the student is performing on a task (Ashford, 1986; Ashford, Blatt, & 
VandeWalle, 2003). Uncertainty is an aversive state that motivates strategies aimed at reducing 
or managing it (Bordia, Hobman, Jones, Gallois, & Callan, 2004). Because uncertainty is often 
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unpleasant and may distract attention away from task performance (Kanfer & Ackennan, 1989), 
reducing uncertainty may lead to higher motivation and more efficient task strategies. 

Second, formative feedback can effectively reduce the cognitive load of learners, 
especially novice or struggling students (e.g., Paas, Renkl, & Sweller, 2003; Sweller, Van 
Merrienboer, & Paas, 1998). These students can become cognitively overwhelmed during 
learning due to high performance demands, and thus they may benefit from supportive feedback 
designed to decrease the cognitive load. In fact, Sweller et al. provided support for this claim by 
showing how the presentation of worked examples reduces the cognitive load for low-ability 
students faced with a complex problem-solving task. Moreno (2004) provided additional support 
using explanatory feedback to support novice learners. 

Finally, feedback can provide information that may be useful for correcting inappropriate 
task strategies, procedural errors, or misconceptions (e.g., Ilgen et al., 1979; Mason & Bruning, 
2001; Mory, 2004; Narciss & Huth, 2004). The corrective function effects appear to be 
especially powerful for feedback that is more specific (Baron, 1988; Goldstein, Emanuel, & 
Howell, 1968), which is described next. 

3.3 Feedback Specificity 

Feedback specificity is defined as the level of information presented in feedback 
messages (Goodman, Wood, & Hendrickx, 2004). In other words, specific (or elaborated) 
feedback provides infonnation about particular responses or behaviors beyond just their accuracy 
and tends to be more directive than facilitative. 

A number of researchers have reported that feedback is significantly more effective when 
it provides details of how to improve the answer, rather than when it just indicates whether the 
student’s work is correct or not (e.g., Bangert-Drowns et al., 1991; Pridemore & Klein, 1995). 
Feedback lacking in specificity may cause students to view it as useless and/or frustrating 
(Williams, 1997). It can also lead to uncertainty about how to respond to the feedback (Fedor, 
1991) and may require greater information processing activity on the part of the learner to 
understand the intended message (Bangert-Drowns et al.). Uncertainty and cognitive load can 
lead to lower levels of learning (Kluger & DeNisi, 1996; Sweller et al., 1998), or even reduced 
motivation to respond to the feedback (Ashford, 1986; Corno & Snow, 1986). 

In an experiment that tested feedback specificity and its relationship to learning, Phye and 
Sanders (1994) tested two types of feedback (i.e., general advice versus specific feedback, the 
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latter providing the learner with the correct answer). Students were assigned to one of the two 
learning conditions, and they received either general advice or specific feedback as part of a 
verbal analogy problem-solving task. In line with the research cited above, Phye and Sanders 
found that the more specific feedback was clearly superior to general advice on a retention task. 
However, they found no significant differences between feedback types on a transfer task. They 
cautioned against assuming that procedures that enhance performance during acquisition (e.g., 
providing specific feedback) will necessarily enhance transfer to new tasks. 

In summary, providing feedback that is specific and clear for conceptual and procedural 
learning tasks is a reasonable, general guideline. However, this practice may depend on other 
variables, such as learner characteristics (e.g., ability level, motivation) and different learning 
outcomes (e.g., retention vs. transfer tasks). In addition, the specificity dimension of formative 
feedback itself is not very specific, as described in the literature. More focused feedback features 
are reviewed in the next section. 

3.4 Features of Formative Feedback 

In an excellent historical review on feedback, Kulhavy and Stock (1989) reported that 
effective feedback provides the learner with two types of infonnation: verification and 
elaboration. Verification is defined as the simple judgment of whether an answer is correct, and 
elaboration is the informational aspect of the message that provides relevant cues to guide the 
learner toward a correct answer. Researchers appear to be converging toward the view that 
effective feedback should include elements of both verification and elaboration (e.g., Bangert- 
Drowns et ah, 1991; Mason & Bruning, 2001). These features are now described in more detail. 

Verification. Confirming whether an answer is correct can be accomplished in several 
different ways. The most common way involves simply stating whether the answer is correct or 
incorrect. With computers, there are more options—some of which are explicit and some more 
implicit. Among explicit verifications, highlighting or otherwise marking a response to indicate 
its correctness (e.g., with a check mark) can convey the information. Implicit verification can 
occur when, for instance, a student’s response yields expected or unexpected results (e.g., within 
a simulation). This review focuses more on explicit than implicit feedback as the former is more 
readily subject to experimental controls. 

Elaboration. Feedback elaboration has even more variations than verification. For 
instance, elaboration can (a) address the topic, (b) address the response, (c) discuss the particular 
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error(s), (d) provide worked examples, or (e) give gentle guidance. The first three types of 
elaborated feedback are more specific and directive, while the last two types are more general 
and facilitative. 

In general, elaborated feedback usually addresses the correct answer and may 
additionally explain why the selected response is wrong and indicate what the correct answer 
should be. There seems to be growing consensus that one type of elaboration, response-specific 
feedback, appears to enhance student achievement, particularly learning efficiency, more than 
other types of feedback, such as simple verification or answer until correct (e.g., Corbett & 
Anderson, 2001; Gilman, 1969; Mory, 2004). However, as will be discussed in Section 3.9 (see 
“Goal Orientation”), feedback specificity has been shown to affect performance by way of an 
interaction with learners’ goal orientations. 

3.5 Feedback Complexity/Length 

While more specific feedback may be generally better than less specific feedback (at least 
under certain conditions), a related dimension to consider in generating feedback is one of length 
or complexity of the infonnation. For example, if feedback is too long or too complicated, many 
learners will simply not pay attention to it, rendering it useless. Lengthy feedback can also 
diffuse or dilute the message. Feedback complexity thus refers to how much and what 
information should be included in the feedback messages. 

Many research articles have addressed the issue of feedback complexity, but only a few 
have attempted to array the major variables along a dimension of complexity (albeit, see 
Dempsey, Driscoll, & Swindell, 1993; Mason & Bruning, 2001; and Narciss & Huth, 2004). I 
have aggregated information from their respective lists into a single compilation (see Table 1), 
arrayed generally from least to most complex infonnation presented. Terms appearing in the 
feedback type column are used throughout the remainder of this paper. 

If formative feedback is to serve a corrective function, even in its simplest form it should 
(a) verify whether the student’s answer is right or wrong and (b) provide information to the 
learner about the correct response (either directive or facilitative). Studies that have examined the 
type and amount of information in feedback, however, have shown inconsistent results (see 
Kulhavy, 1977, and Mory, 2004, for summaries of the range of results). Some specific findings 
on the feedback complexity issue are described below. 
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Table 1 


Feedback Types Arrayed Loosely by Complexity 


Feedback type 

Description 

No feedback 

Refers to conditions where the learner is presented a question and is 
required to respond, but there is no indication as to the 
correctness of the learner’s response. 

Verification 

Also called knowledge of results (KR), or knowledge of outcome, it 
informs the learner about the correctness of her response(s), such 
as right/wrong or overall percentage correct. 

Correct response 

Also known as knowledge of correct response (KCR), it informs 
the learner of the correct answer to a specific problem with no 
additional information. 

Try-again 

Also known as repeat-until-correct feedback, it informs the learner 
about an incorrect response and allows the learner one or more 
attempts to answer the question. 

Error-flagging 

Also known as location of mistakes (LM), error-flagging highlights 
errors in a solution, without giving correct answer. 

Elaborated 

A general tenn, it refers to providing an explanation about why a 
specific response was correct, and it might allow the learner to 
review part of the instruction. It also might present the correct 
answer (see below for six types of elaborated feedback). 

Attribute isolation 

Elaborated feedback that presents infonnation addressing central 
attributes of the target concept or skill being studied. 

Topic-contingent 

Elaborated feedback that provides the learner with information 
relating to the target topic currently being studied. This might 
entail simply re-teaching material. 

Response-contingent 

Elaborated feedback that focuses on the learner’s specific response. 

It may describe why the answer is wrong and why the correct 
answer is correct. This does not use formal error analysis. 

Hints/cues/prompts 

Elaborated feedback that guides the learner in the right direction 
(e.g., strategic hint on what to do next or a worked example or 
demonstration). It avoids explicitly presenting the correct answer. 

Bugs/misconceptions 

Elaborated feedback that requires error analysis and diagnosis. It 
provides information about the learner’s specific errors or 
misconceptions (e.g., what is wrong and why). 

Informative tutoring 

The most elaborated feedback (from Narciss & Huth, 2004), this 
presents verification feedback, error-flagging, and strategic hints 
on how to proceed. The correct answer is not usually provided. 
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No effect of feedback complexity. Schimmel (1983) performed a meta-analysis on 
feedback as used in computer-based instruction and programmed instruction. He analyzed the 
results from 15 experimental studies and found that the amount of information (i.e., feedback 
complexity) was not significantly related to feedback effects. He also found that feedback effects 
were significantly larger in computer-based than in programmed instruction. 

Sleeman et al. (1989) examined conflicting findings in the literature concerning the 
diagnosis and remediation of students’ errors. They noted that few studies have systematically 
compared the effects of different styles of error-based feedback, and of those that have, the 
results are inconclusive. For instance, Swan (1983) found that a conflict approach (pointing out 
errors made by students and demonstrating their consequences, classified in Table 1 as 
bugs/misconceptions) was more effective than re-teaching (classified in Table 1 as topic- 
contingent), but Bunderson and Olsen (1983) found no difference between these two feedback 
approaches. 

To untangle these conflicting findings, Sleeman et al. (1989) conducted three studies that 
explicitly compared error-specific or model-based remediation (MBR, bugs/misconceptions) 
with simply re-teaching the algebra content ( topic-contingent). MBR bases feedback on a model 
of student errors, while re-teaching simply shows students a correct procedure and answer 
without addressing specific errors. Their results showed that MBR (a more complex approach) 
and re-teaching (a simpler approach) are both more effective than no tutoring; however, MBR 
was not more effective than re-teaching. The results are discussed in terms of stability of errors 
and their relevance to educational practice and to intelligent tutoring systems (ITS). Although the 
studies used human tutors, the results suggest that, for procedural skills in algebra, feedback 
based on just re-teaching content was as effective as feedback based on more expensive error 
analyses. 

Negative effects of feedback complexity. Kulhavy, White, Topp, Chan, and Adams (1985) 
similarly examined the feedback complexity issue. They tested a group of college 
undergraduates who read a 2,400-word passage, responded to 16 multiple-choice questions about 
it, and received after responding one of four types of feedback that increased in complexity. 
Feedback complexity was systematically varied. The lowest level was simply verification 
feedback with the correct response, and the most complex included a combination of 
verification, the correct answer, and an explanation about why the incorrect answer was wrong 
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with a pointer to the relevant part of the text passage where the answer could be found. The main 
finding was that complexity of feedback was inversely related to both the ability to correct errors 
and to learn efficiently (i.e., the ratio of feedback study time to posttest score). Specifically, the 
authors showed that more complex versions of feedback had a small effect on students’ ability to 
correct their own errors, and the least complex feedback demonstrated greater learner benefits in 
terms of efficiency and outcome than complex feedback. 

In summary, the inconclusive findings on feedback complexity described above suggest 
that there may be other mediating factors involved in the relationship between formative 
feedback and learning. For instance, instead of feedback complexity, a more salient facet of 
feedback may be the nature and quality of the content, such as providing information about 
learning goals and the attainment of those goals. 

3.6 Goal-Directed Feedback and Motivation 

Goal-directed feedback provides learners with infonnation about their progress toward a 
desired goal (or set of goals), rather than providing feedback on discrete responses (i.e., 
responses to individual tasks). Research has shown that for a learner to remain motivated and 
engaged depends upon a close match between a learner’s goals and expectations that these goals 
can be met (Fisher & Ford, 1998; Ford, Smith, Weissbein, Gully, & Salas, 1998). If goals are set 
so high that they are unattainable, the learner will likely experience failure and become 
discouraged. When goals are set so low that their attainment is certain, success loses its power to 
promote further effort (Bimey, Burdick, & Teevan, 1969). 

According to Malone (1981), goals must have certain features to make them challenging 
for the learner. For example, goals must be personally meaningful and easily generated, and the 
learner must receive perfonnance feedback about whether or not the goals are being attained. 
Hoska (1993) classified goals as being of two types: acquisition (i.e., to help the learner acquire 
something desirable) and avoidance (i.e., to help the learner avoid something undesirable). 
Moreover, acquisition and avoidance goals can be either external or internal. 

Motivation has been shown to be an important mediating factor in learners’ performance 
(Covington & Omelich, 1984), and feedback can be a powerful motivator when delivered in 
response to goal-driven efforts. Some researchers suggest that the learner’s goal orientation 
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should be considered when designing instruction, particularly when feedback can encourage or 
discourage a learner’s effort (Dempsey et ah, 1993). Goal orientation describes the manner in 
which people are motivated to work toward different kinds of goals. The idea is that individuals 
hold either a learning or performance orientation toward tasks (e.g., Dweck, 1986). A learning 
orientation is characterized by a desire to increase one’s competence by developing new skills 
and mastering new situations with the belief that intelligence is malleable. In contrast, 
performance orientation reflects a desire to demonstrate one’s competence to others and to be 
positively evaluated by others, with the belief that intelligence is innate (Farr, Hofmann, & 
Ringenbach, 1993). 

Research has shown that the two types of goal orientation differentially influence how 
individuals respond to task difficulty and failure (Dweck & Leggett, 1988). That is, individuals 
with a learning orientation are characterized by persistence in the face of failure, the use of more 
complex learning strategies, and the pursuit of challenging material and tasks. Performance 
orientation is characterized by a tendency to withdraw from tasks (especially in the face of 
failure), less interest in difficult tasks, and the tendency to seek less challenging material and 
tasks on which success is likely. Consistent with these labels, research has generally shown that 
learning orientation is associated with more positive outcomes and performance orientation is 
related to either equivocal or negative outcomes (e.g., Button, Mathieu, & Zajac, 1996; Fisher & 
Ford, 1998; VandeWalle, Brown, Cron, & Slocum, 1999). 

One way to influence learners’ goal orientations (e.g., to shift from a focus on performing 
to an emphasis on learning) is via formative feedback. Hoska (1993) showed how goal- 
orientation feedback can modify a learner’s view of intelligence, by helping a learner see that (a) 
ability and skill can be developed through practice, (b) effort is critical to increasing this skill, 
and (c) mistakes are part of the skill-acquisition process. Feedback can also serve as a cognitive 
support mechanism, described next. 

3 .7 Formative Feedback as Scaffolding 

Like training wheels, scaffolding enables learners to do more advanced activities and to 
engage in more advanced thinking and problem solving than they could without such help. 
Eventually, high level functions are gradually turned over to the students as the teacher (or 
computer system) removes the scaffolding and fades away from the process (see Collins, Brown, 
& Newman, 1989; Graesser, McNamara, & VanLehn, 2005). For instance, Graesser, McNamara, 
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and VanLehn described a theoretically based approach to facilitating explanation-centered 
learning via scaffolding, including (a) pedagogical agents that scaffold strategies, metacognition, 
and explanation construction; (b) computer coaches that facilitate answer generation to questions 
that require explanations by using mixed-initiative dialogue; and (c) modeling and coaching 
students in constructing self-explanations. Their systems (i.e., Point&Query, AutoTutor, and 
iSTART) that were built with these components have shown promising results in tests of 
learning gains and improved learning strategies. 

In their book, How People Learn, Bransford, Brown, and Cocking (2000) described how 
psychological theories and insights can be translated into actions and practices. In relation to 
feedback, they suggested a goal-directed approach to learning using scaffolding (or scaffolded 
feedback) that (a) motivates the learner’s interest related to the task, (b) simplifies the task to 
make it more manageable and achievable, (c) provides some direction to help the learner focus 
on achieving the goal, (d) clearly indicates the differences between the learner’s work and the 
standard or desired solution, (e) reduces frustration and risk, and (f) models and clearly defines 
the expectations (goals) of the activity to be perfonned. 

Conventional wisdom suggests that facilitative feedback (providing guidance and cues, as 
illustrated in the research cited above) would enhance learning more than directive feedback 
(providing corrective infonnation), yet this is not necessarily the case. In fact, some research has 
shown that directive feedback may actually be more helpful than facilitative—particularly for 
learners who are just learning a topic or content area (e.g., Knoblauch & Brannon, 1981; 

Moreno, 2004). Because scaffolding relates to the explicit support of learners during the learning 
process, scaffolded feedback in an educational setting may include models, cues, prompts, hints, 
partial solutions, as well as direct instruction (Hartman, 2002). Scaffolding is gradually removed 
as students gain their cognitive footing, thus directive feedback may be most helpful during the 
early stages of learning. Facilitative feedback may be more helpful later on, and the question is 
when. According to Vygotsky (1987), external scaffolds can be removed when the learner 
develops more sophisticated cognitive systems, where the system of knowledge itself becomes 
part of the scaffold for new learning. The issue of feedback timing is discussed in more detail in 
the following section. 
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3.8. Timing 

It was my teacher's genius, her quick sympathy, her loving tact which made the first years 

of my education so beautiful. It was because she seized the right moment to impart 

knowledge that made it so pleasant and acceptable to me. -Helen Keller 

Similar to the aforementioned feedback variables (e.g., complexity and specificity), there 
are also conflicting results in the literature relating to the timing of feedback and the effects on 
learning outcome and efficiency. Researchers have been examining the effects of immediate 
versus delayed feedback on learning for decades (e.g., Clariana, 1999; Jurma & Froelich, 1984; 
Pound & Bailey, 1975; Prather & Berry, 1973; Reddy, 1969). The timing-of-feedback literature 
concerns whether feedback should be delivered immediately or delayed. Immediately may be 
defined as right after a student has responded to an item or problem or, in the case of summative 
feedback, right after the student has completed a quiz or test. Delayed is usually defined relative 
to immediately, and such feedback may occur minutes, hours, weeks, or longer after the student 
completes some task or test. 

Regardless of the particular unit of time, the effects of the feedback timing variable are 
mixed. Again, while there appears to be no consistent main effect of timing, there are 
interactions involving the timing of feedback and learning. Some researchers have argued for 
immediate feedback as a way of preventing errors being encoded into memory, while others have 
argued that delayed feedback reduces proactive interference, thus allowing the initial error to be 
forgotten and the correct information to be encoded with no interference (for more on this 
debate, see Kulhavy & Anderson, 1972). 

Support for delayed feedback. Researchers who support using delayed feedback generally 
adhere to what is called the interference-perseveration hypothesis proposed by Kulhavy and 
Anderson (1972). This asserts that initial errors do not compete with to-be-learned correct 
responses if corrective information is delayed. This is because errors are likely to be forgotten 
and thus cannot interfere with retention. 

The superiority of delayed feedback, referred to as the delay-retention effect (DRE), was 
supported in a series of experiments by Anderson and his colleagues (e.g., Kulhavy & Anderson, 
1972; Surber & Anderson, 1975) that compared the accuracy of responses on a retention test 
with the accuracy of responses on an initial test. Although there are many studies in the literature 
that do not support the delay-retention effect (e.g., Kippel, 1974; Newman, Williams, & Hiller, 
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1974; Phye & Bailer, 1970), delayed feedback has often been shown to be as effective as 
immediate feedback. 

Schroth (1992) presented the results from an experiment that investigated the effects of 
delayed feedback and type of verbal feedback on transfer, using a concept formation task. The 
four conditions of delayed feedback were: 0 seconds, 10 seconds, 20 seconds, and 30 seconds. 
The verbal feedback conditions were (a) correct-incorrect (verification feedback), (b) correct- 
nothing (i.e., where nothing means that no feedback was presented if the student solved an item 
incorrectly), and (c) nothing-incorrect (i.e., no feedback was presented if the student answered 
correctly). All participants were tested seven days following an initial learning trial. The finding 
relevant to this paper is that although delayed feedback slowed down the rate of initial learning, 
it facilitated transfer after the delay. 

Support for immediate feedback. Supporters of immediate feedback theorize that the 
earlier corrective infonnation is provided, the more likely it is that efficient retention will result 
(Phye & Andre, 1989). The superiority of immediate over delayed feedback has been 
demonstrated for the acquisition of verbal materials, procedural skills, as well as some motor 
skills (Anderson, Magill, & Sekiya, 2001; Brosvic & Cohen, 1988; Corbett & Anderson, 1989; 
2001; Dihoff, Brosvic, Epstein, & Cook, 2003). 

Corbett and Anderson (2001) have been using immediate feedback successfully in their 
programming and mathematics tutors for almost two decades (see Anderson, Corbett, Koedinger, 
& Pelletier, 1995). For instance, they used their ACT Programming Tutor to examine differential 
timing effects on students’ learning. The study involved four feedback conditions, the first three 
of which offered the student different levels of control over error feedback and correction: (a) 
immediate feedback and immediate error correction (i.e., the tutor intervened as soon as students 
made errors and forced them to correct the error before moving on), (b) immediate error-flagging 
and student control of error correction, (c) feedback on demand and student control of error 
correction, and (d) no-tutor condition and no step-by-step, problem-solving support (the control 
condition). The immediate feedback group with greatest tutor control of problem solving yielded 
the most efficient learning (i.e., condition a). These students completed the tutor problems 
fastest, and their perfonnance on criterion tests was equivalent to that of the other groups 
(excluding the control group). Furthermore, questionnaires showed no significant differences in 
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terms of preference among the tutor conditions. This study demonstrated that immediate error 
feedback helped with immediate learning. 

Azevedo and Bernard (1995) conducted a meta-analysis on the literature concerning the 
effects of feedback on learning from computer-based instruction (CBI). They noted that despite 
the widespread acceptance of feedback in computerized instruction, empirical support for 
particular types of feedback infonnation has been inconsistent and contradictory. Effect size 
calculations were performed on 22 CBI studies comparing feedback versus no-feedback relating 
to immediate outcomes. This resulted in a mean weighted effect size of 0.80. The results from 9 
studies employing delayed outcome conditions resulted in a mean weighted effect size of just 
0.35. This provides support for the strength of feedback in relation to immediate outcome 
administrations, at least in CBI. 

Conjoining feedback timing findings. A preliminary conclusion derived from both the 
Schroth (1992) and Corbett and Anderson (2001) findings is that delayed feedback may be 
superior for promoting transfer of learning, especially in relation to concept formation tasks, 
while immediate feedback may be more efficient, particularly in the short run and for procedural 
skills (i.e., programming and mathematics). This proposition has some support. For instance, 
Schmidt, Young, Swinnen, and Shapiro (1989) conducted an experiment that provided 
verification feedback following a set of trials relating to a relatively simple ballistic-timing task. 
Feedback timing consisted of one of four lengths: 1 (i.e., verification after every trial), 5 
(verification after 5 trials), 10, and 15 trials. During the acquisition phase when feedback was 
present, all groups showed general improvements in perfonnance across practice, although those 
in the longer length conditions showed worse performance relative to the shorter length 
conditions. In a delayed test, Schmidt et al. found an inverse relation between the timing variable 
(1,5, 10, or 15 trials between feedback) and error rates. That is, longer delays between feedback 
episodes resulted in relatively poorer performance during acquisition, but better retention 
compared with shorter delay conditions. 

Mathan and Koedinger (2002) reviewed various studies on the timing of feedback and 
concluded that the effectiveness of feedback depends not on the main effect of timing, but on the 
nature of the task and the capability of the learner. They called for further exploration on 
possible interactions involving timing effects and optimal ways to match feedback (type and 
timing) to learning tasks and students’ individual needs and/or characteristics (e.g., Schimmel, 
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1988; Smith & Ragan, 1999). One such interaction reported in the literature concerns feedback 
timing and task difficulty. That is, if the task is difficult, then immediate feedback is beneficial, 
but if the task is easy, then delayed feedback may be preferable (Clariana, 1999). This is similar 
to the ideas presented earlier in Section 3.7. 

Summary of feedback timing results. Research investigating the relationship of feedback 
timing and learning/performance reveals inconsistent findings. 1 One way to resolve the 
inconsistency is by considering that immediate feedback may activate both positive and negative 
learning effects. For instance, the positive effects of immediate feedback can be seen as 
facilitating the decision or motivation to practice and providing the explicit association of 
outcomes to causes. On the downside, immediate feedback may lead to reliance on information 
that is not available during transfer, and it also may promote less careful or mindful behavior. If 
this supposition is true, then the positive and negative effects of immediate feedback could 
potentially cancel each other out. Alternatively, either the positive or negative effects may come 
to the fore, depending on the experimental context. A similar argument could be made for 
delayed feedback effects on learning. For example, on the positive side, delayed feedback may 
encourage learners’ engagement in active cognitive and metacognitive processing, thus 
engendering a sense of autonomy (and perhaps improved self-efficacy). But on the negative side, 
delaying feedback for struggling and less motivated learners may prove to be frustrating and 
detrimental to their knowledge and skill acquisition. 

3.9 Feedback and Other Variables 

So far, formative feedback types and timing have been discussed in relation to their 
effects on learning. This section examines other variables that may interact with feedback 
features, such as learner-ability level, response certitude, goal orientation, and normative 
feedback. 

Learner level. As alluded to in Section 3.8, some research has suggested that low 
achieving students may benefit from immediate feedback, while high achieving students may 
prefer or benefit from delayed feedback (Gaynor, 1981; Roper, 1977). Furthermore, when testing 
different types of feedback, Clariana (1990) argued that low ability students benefit from 
correct-response feedback more than from try-again feedback. Hanna (1976) also examined 
student performance in relation to different feedback conditions: verification, elaboration, and 
no feedback. Verification feedback produced the highest scores for high-ability students, while 
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elaborated feedback produced the highest scores for low ability students. There were no 
significant differences between verification and elaborated feedback for middle ability students, 
but both of these types of feedback were superior to no feedback. These findings support the 
research and suppositions presented earlier in Section 3.7. 

Response certitude. Kulhavy and Stock (1989) examined feedback and response certitude 
issues from an infonnation processing perspective. That is, they had students provide confidence 
judgments (response certitude ratings) following each response to various tasks. They 
hypothesized that when students are certain that their answer is correct, they will spend little time 
analyzing feedback, and when students are certain that their answer is incorrect, they will spend 
more time reviewing feedback. The implications of this are straightforward; that is, provide more 
elaborated feedback for students who are more certain that their answer is wrong, and deliver 
more constrained feedback for those with high certitude of correct answers. While Kulhavy and 
Stock’s research supported their hypotheses, other studies did not replicate the findings. For 
instance, Mory (1994) tried to replicate the response-certitude findings and found that while 
there were differences in the amount of time needed to study the feedback, there was no 
significant learning effect for feedback tailored to response certitude and correctness. 

Goal orientation. Davis, Carson, Ammeter, and Treadway (2005) reported the results of a 
study testing the relationship between goal orientation and feedback specificity on performance 
using a management decision-making task. (Feedback specificity was discussed in Section 3.3.) 
In short, they found that feedback specificity (low, moderate, and high levels) had a significant 
influence on performance for individuals who were low on learning orientation (i.e., high 
feedback specificity was better for learners with low learning orientation). They also reported a 
significant influence of feedback specificity on performance for persons high in performance 
orientation (i.e., this group also benefited from more specific feedback). The findings support the 
general positive effects of feedback on performance and further suggest the use of more specific 
feedback for learners with either high perfonnance or low learning goal orientations. 

Normative feedback. According to research cited in Kluger and DeNisi (1996), when 
feedback is provided to students in a nonn-referenced manner that compares the individual's 
performance to that of others, people who perform poorly tend to attribute their failures to lack 
of ability, expect to perfonn poorly in the future, and demonstrate decreased motivation on 
subsequent tasks (i.e., similar to learners with a performance orientation, as described earlier). 
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McColskey and Leary (1985) examined the hypothesis that the harmful effects of failure might 
be lessened when failure is expressed in self-referenced terms (i.e., relative to the learner’s 
known level of ability as assessed by other measures). In their study, learners received feedback 
indicating that they did well or poorly on an anagram test, and this feedback was described as 
either norm-referenced (comparing the individual’s performance to that of others) or as self- 
referenced (comparing perfonnance to other measures of the individual’s ability). They found 
that, compared to nonn-referenced feedback, self-referenced feedback resulted in higher 
expectancies regarding future performance and increased attributions to effort (e.g., “I succeeded 
because I worked really hard”). Attributions to ability (e.g., “I succeeded because I’m smart”) 
were not affected. The main implication is that low achieving students should not receive 
nonnative feedback but instead, self-referenced feedback—focusing their attention on their own 
particular progress. 

This review has presented research findings covering the gamut of formative feedback 
variables. As with earlier reviews, this one has unearthed mixed findings regarding learning 
effects—whether examining feedback specificity, timing of feedback, and so on. The next 
section presents four influential feedback research papers that have attempted to integrate 
disparate findings into preliminary theories (or models) via large literature reviews and/or meta¬ 
analyses. The papers summarized in this section include Kluger and DeNisi (1996), Bangert- 
Drowns et al. (1991), Narciss and Huth (2004), and Mason and Bruning (2001). 

4. Toward a Framework of Formative Feedback 

To understand the world, one must not be worrying about one’s self. -Einstein 

4.1 Kluger and DeNisi (1996) 

Ten years ago, Kluger and DeNisi (1996) examined and reported on the effects of 
feedback interventions (FIs) on performance from multiple perspectives and spanning decades of 
research—back to Thorndike’s classic research in the early 1900s. Kluger and DeNisi conducted 
an extensive review of the literature, performed a meta-analysis on reported experimental 
findings, and constructed a preliminary theory based on a number of variables, or moderators. 
Their preliminary FI theory (FIT) offers a broad approach to investigating FI effects, including 
such feedback moderators as praise, written or verbal feedback, task novelty and complexity, 
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time constraints, and types of tasks such as physical, memory, knowledge, and vigilance tasks. 
The basic premise underlying FIT is that FIs change the locus of a learner’s attention among 
three levels of control: (a) task-learning, (b) task-motivation, and (c) meta-task processes (see 
Figure 1). 



Figure 1. Abstract hierarchy of processing. 

The general pattern of results from their large meta-analysis of FI studies was consistent 
with the argument that all else being equal, FI cues affect performance by changing the locus of 
attention. The lower in the hierarchy the FI-induced locus of attention is, the stronger the benefit 
of an FI for perfonnance. In other words, formative feedback that focuses the learner on aspects 
of the task (i.e., the lower part of Figure 1) promotes learning and achievement compared to FIs 
that draw attention to the self (i.e., the upper box in Figure 1), which can impede learning. This 
phenomenon was described earlier. 

FIT consists of five basic arguments: (a) behavior is regulated by comparisons of 
feedback to goals or standards, (b) goals or standards are organized hierarchically, (c) attention is 
limited and therefore only feedback-standard gaps (i.e., discrepancies between actual and desired 
performance) that receive attention actively participate in behavior regulation, (d) attention is 
normally directed to a moderate level of the hierarchy, and (e) FIs change the locus of attention 
and therefore affect behavior. These arguments are interdependent, and each consecutive 
argument is built on the preceding argument. 
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Specific results from Kluger and DeNisi’s (1996) meta-analysis showed that four 
moderators (feedback variables) demonstrated significant relationships with d (effect size) at the 
p < 0.01 level: (a) discouraging feedback interventions (FIs) reduce FI effects, (b) velocity FIs 
(i.e., self-referenced feedback that addresses a change from the learner’s prior performance), (c) 
correct response FIs increase effects, and (d) FI effects on performance of physical tasks are 
lower than FI effects on cognitive tasks. 

Six more moderators became significant after excluding biased studies from the meta¬ 
analysis. Of those six, three of them were shown to reduce FI effects: (a) praise, (b) FIs that 
threaten self-esteem, and (c) orally delivered FIs (from the instructor). Feedback interventions 
that provide frequent messages enhance FI effects, and FI effects are stronger for memory tasks 
and weaker for more procedural tasks. Finally, other variables showing significance at p < 0.05 
level include the following: computerized FIs, which yield stronger effects than 
noncomputerized FIs; FIs in the context of complex tasks, which yield weaker effects than for 
simpler tasks; and FIs with a goal-setting intervention, which are more effective than in the 
absence of goal setting. Figure 2 summarizes the main findings. This figure represents my 
interpretation (and categorization) of data presented in Kluger and DeNisi (1996). 

One important finding from these results concerns the attenuating effect of praise on 
learning and performance, although this has been described elsewhere in the literature in terms of 
a model of self-attention (Baumeister, Hutton, & Cairns, 1990), attributions of effort (Butler, 
1987), and control theory (Waldersee & Luthans, 1994). Also, Balcazar, Hopkins, & Suarez 
(1985) reported that praise was not as widely effective a rein forcer as previously believed. 

Perhaps the most surprising finding that emerged from the Kluger and DeNisi (1996) 
meta-analysis is that in over one third of the 607 cases (effect sizes), FIs reduced perfonnance. 
Furthermore, most of the observed variability cannot be explained by sampling or other errors. 

In conclusion, and as the authors observe in a later paper on the topic (Kluger & DeNisi, 
1998), FIs may be viewed as double-edged swords. Care should be taken to know which 
interventions increase performance and under which particular conditions. 


22 




Figure 2. FI moderators and their relationships to learning/performance. 


4.2 Bangert-Drowns, Kulik, Kulik, and Morgan (1991) 

Bangert-Drowns et al. (1991) examined 40 research studies on feedback using meta¬ 
analysis techniques. They examined such variables as type of feedback, timing of feedback, and 
error rates in terms of their respective effect sizes. This widely cited article describes both 
behavioral and cognitive operations that occur in learning. The basic idea is that in order to direct 
behavior, a learner needs to be able to monitor physical changes brought about by the behavior. 
That is, learners change cognitive operations and thus activity by adapting to new information 
and matching the results of activities to their expectations about performance. Bangert-Drowns et 
al. emphasized that, “any theory that depicts learning as a process of mutual influence between 
learners and their environments must involve feedback implicitly or explicitly because, without 
feedback, mutual influence is by definition impossible. Hence, the feedback construct appears 
often as an essential element of theories of learning and instruction” (p. 214). To make this point 
more concrete, imagine trying to learn something new in the absence of any feedback (explicit or 
implicit). 
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Most of the variables they analyzed comprised text-based feedback, which they organized 
into a five-stage model. This model describes the state of learners as they move through a 
feedback cycle and emphasizes the construct of mindfulness (Salomon & Globerson, 1987). 
Mindfulness is “a reflective process in which the learner explores situational cues and underlying 
meanings relevant to the task involved” (Dempsey et ah, 1993, p. 38). 

The five stages are depicted in Figure 3 and are similar to other learning cycles (e.g., 
Kolb, 1984, and Gibbs, 1988), particularly in relation to the importance of reflection. 



Figure 3. Five-stage model of the learner during a feedback cycle. 

As described by Bangert-Drowns et al. (1991, p. 217), the five states of the learner 
receiving feedback include: 

1. The initial or current state of the learner. This is characterized by the degree of 
interest, goal orientation, degree of self-efficacy, and prior relevant knowledge. 

2. Search and retrieval strategies. These cognitive mechanisms are activated by a 
question. Infonnation stored in the context of elaborations would be easier to locate in 
memory due to more pathways providing access to the information. 

3. The learner makes a response to the question. In addition, the learner feels some 
degree of certainty about the response and thus has some expectation about what the 
feedback will indicate. 

4. The learner evaluates the response in light of information from the feedback. The 
nature of the evaluation depends on the learner’s expectations about feedback. For 
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instance, if the learner was sure of the response and the feedback confirmed its 
correctness, the retrieval pathway may be strengthened or unaltered. If the learner was 
sure of the response and feedback indicated its incorrectness, the learner may seek to 
understand the incongruity. Uncertainty about a response with feedback confirmation 
or disconfirmation is less likely to stimulate deep reflection unless the learner was 
interested in acquiring the instructional content. 

5. Adjustments are made to relevant knowledge, self-efficacy, interests, and goals as a 
result of the response evaluation. These adjusted states, with subsequent experiences, 
determine the next current state. 

Overall, the meta-analysis by Bangert-Drowns et al. (1991) found generally weak effects 
of feedback on achievement. More specifically (but not surprisingly), the authors found that 
verification feedback (correct/incorrect) resulted in lower effect sizes compared to correct 
response feedback (i.e., providing the correct answer). Also, using a pretest within a study 
significantly lowered effect sizes, as did uncontrolled presearch availability of answers (i.e., 
ability to locate an answer prior to responding to a question). These last two findings may be 
because pretests and presearch availability may be seen as advance organizers, which may 
support short-term retention, but undennine overall feedback effects in studies that employ them. 

The main conclusion from the meta-analysis by Bangert-Drowns et al. (1991) and 
subsequent five-cycle model is that feedback can promote learning if it is received mindfully. 
Conversely, feedback can inhibit learning if it encourages mindlessness, as when the answers are 
made available before learners begin their memory search, or if the feedback message does not 
match students’ cognitive needs (e.g., too easy, too complex, too vague). 

4.3 Narciss and Huth (2004) 

Narciss and Huth (2004) outlined a conceptual framework for the design of formative 
feedback. This framework is based on the body of research relating to elaborated feedback types. 
Cognitive task and error analyses served as the basis for the design of the framework. The impact 
of the feedback on learning and motivation was ultimately examined in two computer-based 
learning experiments. The results of these studies showed that systematically designed formative 
feedback has positive effects on achievement and motivation. 
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In general, they assert that designing and developing effective formative feedback needs 
to take into consideration instructional context, as well as characteristics of the learner, to 
provide effective feedback for complex learning tasks. The conceptual framework for the design 
of formative feedback is depicted in Figure 4, based on information presented in Narciss and 
Huth (2004). 


Objectives PtiCK ksa 1 

| (A Taste Errors Objectives Motivation 

Instruction Learner 

Function 

Content Presentation) 

Feedback 

Figure 4. Factors interacting with feedback to influence learning. 

Each of the three factors is examined in more detail below. 

• Feedback. The feedback factor consists of three main elements: (a) the content of the 
feedback (i.e., evaluative aspects, such as verification, as well as informative aspects, 
such as hints, cues, analogies, explanations, and worked out examples); (b) the 
function of the feedback (i.e., cognitive, metacognitive, and motivational); and (c) the 
presentation of the feedback components (i.e., timing, schedule, and perhaps 
adaptivity considerations). 

• Instruction. The instructional factor or context also consists of three main elements: 
(a) the particular instructional objectives (e.g., learning goals or standards relating to 
some curriculum), (b) the learning tasks (e.g., knowledge items, cognitive operations, 
and metacognitive skills), and (c) errors and obstacles (e.g., typical errors, incorrect 
strategies, and sources of errors). 

• Learner. Information concerning the learner that is relevant to feedback design 
includes: (a) learning objectives and goals; (b) prior knowledge, skills, and abilities 
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(e.g., domain-dependent, such as content knowledge, and domain-independent, such 
as metacognitive skills); and (c) academic motivation (e.g., one’s need for academic 
achievement, academic self-efficacy, and meta-motivational skills). 

The authors contend that adapting the content, function, and presentation format of the 
feedback message should be driven by considerations of the instructional goals and learner 
characteristics to maximize the infonnative value of the feedback. Specific steps for generating 
effective formative feedback include selecting and specifying learning objectives (concrete 
learning outcomes), identifying learning tasks, matching to learning outcomes, and after 
conducting cognitive task and error analyses, specifying information (i.e., formative feedback) 
that addresses specific, systematic errors or obstacles. 

4.4 Mason and Bruning (2001) 

Mason and Bruning (2001) reviewed the literature on feedback that is delivered via 
computer-based instructional systems and presented a theoretical framework intended to help 
designers, developers, and instructors build their own computer-based instructional tools. Mason 
and Bruning’s theoretical framework, depicted in Figure 5, is based on research that has 
examined types of feedback and levels of elaboration in relation to student achievement level, 
task complexity, timing of feedback, and prior knowledge. The general recommendation they 
have drawn from the framework is that immediate feedback for students with low achievement 
levels in the context of either simple (lower level) or complex (higher level) tasks is superior to 
delayed feedback, while delayed feedback is suggested for students with high achievement 
levels, especially for complex tasks. 

The following research supports their framework. First, significant learning gains often 
show up in response to various types of elaboration feedback (e.g., Clariana, 1990; Pridemore & 
Klein, 1991, 1995; Morrison, Ross, Gopalakrishnan, & Casey, 1995; Roper, 1977; Waldrop, 
Justen, & Adams, 1986). Second, research conducted in classroom settings seems to suggest that 
response-contingent feedback enhances student achievement more than other types of feedback 
(e.g., Whyte, Karolick, Neilsen, Elder, & Hawley, 1995). Third, they report that the level of 
feedback complexity (see Section 3.5) has been shown to both influence and not influence 
learning, and this lack of effect may be due to interactions involving other variables, such as the 
nature of the topic (e.g., memorizing vs. troubleshooting) and the type of skill measured (e.g., 
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Hodes, 1985; Park & Gittelman, 1992). In cases where the level of feedback complexity has been 
shown to affect learning, more elaborative information tends to produce increased understanding 
(Gilman, 1969; Pridemore & Klein, 1991; Roper 1997; Waldrop et ah; Whyte et ah). For 
instance, while verification feedback did not improve learning, correct response, response- 
contingent, and a combination of the other levels of feedback have been shown to significantly 
improve student learning (e.g., Gilman, 1969). This may be due to the extra information 
available in elaboration feedback, which allows students to correct their own errors or 
misconceptions. Information on the correctness of an answer (i.e., verification feedback) does 
not have much utility for learning. 



Figure 5. Feedback variables for decision making in computer-based instruction. 


Note. Adapted from “Providing Feedback in Computer-Based Instruction: What the Research 
Tells Us” by B. J. Mason and R. Bruning, 2001, Center for Instructional Innovation, University of 
Nebraska-Lincoln. Copyright 2001 by B. J. Mason and R. Bruning. Reprinted with pennission. 
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5. Summary and Discussion 


I'm trying to free your mind, Neo. But I can only show you the door. You’re the one who 

must walk through it. — The Matrix (1999) 

Morgan (2006) likened formative feedback to a good murder in that effective and useful 
feedback depends on three things: (a) motive (the student needs it), (b) opportunity (the student 
receives it in time to use it), and (c) means (the student is able and willing to use the feedback). 
However, even with motive, opportunity, and means, there is still large variability of feedback 
effects on performance and learning, including negative findings that have historically been 
ignored in the literature (see Kluger & DeNisi, 1996). 

Despite this variability, several meta-analyses found that feedback generally improves 
learning, ranging from about .40 of a standard deviation (Guzzo, Jette, & Katzell, 1985) to .80 
and higher (Azevedo & Bernard, 1995; Kluger & DeNisi, 1996), compared to control conditions. 
But major gaps remain in the feedback literature, particularly in relation to interactions among 
task characteristics, instructional contexts, and student characteristics that potentially mediate 
feedback effects. So while there is no simple answer to what feedback works, some preliminary 
guidelines can be formulated based on the findings reported in this review. 

5.1 Recommendations and Guidelines for Formative Feedback 

Tables 2-5 present suggestions or prescriptions based on the current review of the 
formative feedback literature. These are intended to provide a point of departure for more 
comprehensive and systematic prescriptions in the future. Equivocal findings are not presented, 
and the references are not exhaustive, but representative. The tables differ in terms of formative 
feedback guidelines for: (a) things to do, in Table 2, (b) things to avoid, in Table 3, (c) timing 
issues, in Table 4, and (d) learner characteristics, in Table 5. 
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Table 2 


Formative Feedback Guidelines to Enhance Learning (Things to Do) 



Prescription 

Description and references 

1 

Focus feedback on the task, 
not the learner. 

Feedback to the learner should address specific features of the 
learner’s work in relation to the task, with suggestions on how to 
improve (e.g., Butler, 1987; Corbett & Anderson, 2001; Kluger & 
DeNisi, 1996; Narciss & Fluth, 2004). 

2 

Provide elaborated 
feedback to enhance 
learning. 

Feedback should describe the what, how, and/or why of a given 

problem. This type of cognitive feedback is typically more effective 
than verification of results (e.g., Bangert-Drowns et al., 1991; 

Gilman, 1969; Mason & Bruning, 2001; Narciss & Fluth, 2004; 

Shute, 2006). 

3 

Present elaborated feedback 
in manageable units. 

Provide elaborated feedback in small enough pieces so that it is not 
overwhelming and/or discarded (Bransford et al., 2000; Sweller et 
al., 1998). Presenting too much information may not only result in a 
superficial learning, but also invoke cognitive overload (e.g., Mayer 
& Moreno, 2002; Phye & Bender, 1989). A stepwise presentation of 
feedback offers the possibility to control for mistakes and gives 
learners sufficient information to correct errors on their own. 

4 

Be specific and clear with 
feedback messages. 

If feedback is not specific or clear, it can impede learning and can 
frustrate learners (e.g., Moreno, 2004; Williams, 1997). If possible, 
try to link feedback clearly and specifically to goals and 
performance (Floska, 1993; Song & Keller, 2001). 

5 

Keep feedback as simple as 
possible but no simpler 
(based on learner needs and 
instructional constraints). 

Simple feedback is generally based on one cue (e.g., verification or 
hint) and complex feedback on multiple cues (e.g., verification, 
correct response, error analysis). Keep feedback as simple and 
focused as possible. Generate only enough information to help 
students and not more. Kulhavy et al. (1985) found that feedback 
that was too complex did not promote learning compared to simpler 
feedback. 

6 

Reduce uncertainty between 
performance and goals. 

Formative feedback should clarify goals and seek to reduce or remove 
uncertainty in relation to how well learners are performing on a task 
and what needs to be accomplished to attain the goal(s) (e.g., 

Ashford, Blatt, & VandeWalle, 2003; Bangert-Drowns et al., 1991). 

7 

Give unbiased, objective 
feedback, written or via 
computer. 

Feedback from a trustworthy source will be considered more seriously 
than other feedback, which may be disregarded. This may explain 
why computer-based feedback is often better than human-delivered 
in some experiments in that perceived biases are eliminated (see 
Kluger & DeNisi, 1996). 

8 

Promote a learning goal 
orientation via feedback. 

Formative feedback can be used to alter goal orientation—from a 
focus on performance to a focus on learning (Floska, 1993). This 
can be facilitated by crafting feedback emphasizing that effort yields 
increased learning and performance and that mistakes are an 
important part of the learning process (Dweck, 1986). 

9 

Provide feedback after 
learners have attempted a 
solution. 

Do not let learners see answers before trying to solve a problem on 
their own (i.e., presearch availability). Several studies that have 
controlled presearch availability show a benefit of feedback, while 
studies without such control show inconsistent results (Bangert- 
Drowns et al., 1991). 
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Table 3 


Formative Feedback Guidelines to Enhance Learning (Things to Avoid) 



Prescription 

Description and references 

10 

Do not give normative 
comparisons. 

Feedback should avoid comparisons with other students—directly 
or indirectly (e.g., grading on the curve). In general, do not draw 
attention to self during the course of learning (Kluger & DeNisi, 
1996; Wiliam, in press). 

11 

Be cautious about providing 
overall grades. 

Feedback should note areas of strength and provide information on 
how to improve, as warranted and without overall grading. 

Wiliam (in press) summarized the following findings: (a) 
students receiving just grades showed no learning gains, (b) 
those getting just comments showed large gains, and (c) those 
with grades and comments showed no gains (likely due to 
focusing on the grade and ignoring comments). Effective 
feedback relates to the content of the comments (Butler, 1987; 
McColskey & Leary, 1985). 

12 

Do not present feedback that 
discourages the learner or 
threatens the learner’s self¬ 
esteem. 

This prescription is based not only on common sense, but also 
research reported in Kluger and DeNisi (1996), which reports 
feedback interventions that undermine learning as it draws focus 
to the self and away from the task at hand. In addition, do not 
provide feedback that is either too controlling or critical of the 
learner (Baron, 1993; Fedor, Davis, Maslyn, & Mathieson, 

2001). 

13 

Use praise sparingly, if at all. 

Kluger & DeNisi (1996), Butler (1987), and others have noted that 
use of praise as feedback directs the learner’s attention to self, 
which distracts from the task and consequently from learning. 

14 

Try to avoid delivering 
feedback orally. 

This also was addressed in Kluger & DeNisi (1996). When 
feedback is delivered in a more neutral manner (e.g., written or 
computer-delivered), it is construed as less biased. 

15 

Do not interrupt the learner 
with feedback if the learner is 
actively engaged. 

Interrupting a student who is immersed in a task—trying to solve a 
problem or task—can be disruptive to the student and impede 
learning (Corno & Snow, 1986). 

16 

Avoid using progressive hints 
that always terminate with the 
correct answer. 

While hints can be facilitative, they can also be abused. If they are 
employed to scaffold learners, make provisions to prevent their 
abuse (e.g., Aleven & Koedinger, 2000; Shute, Woltz, & Regian, 
1989). Consider using prompts and cues (i.e., more specific 
kinds of hints). 

17 

Do not limit the mode of 
feedback presentation to text. 

Exploit the potential of multimedia to avoid cognitive overload due 
to modality effects (e.g., Mayer & Moreno, 2002). Do not default 
to presenting feedback messages as text. Instead, consider 
alternative modes of presentation (e.g., acoustic, visual). 

18 

Minimize use of extensive 
error analyses and diagnosis. 

The cost of conducting extensive error analyses and cognitive 
diagnosis may not provide sufficient benefit to learning (Sleeman 
et al., 1989; VanLehn et al., 2005). Furthermore, error analyses 
are rarely complete and not always accurate, thus only are 
helpful in a subset of circumstances. 
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Table 4 

Formative Feedback Guidelines in Relation to Timing Issues 


Prescription 

19 Design timing of feedback 
to align with desired 
outcome. 


20 For difficult tasks, use 
immediate feedback. 


21 For relatively simple tasks, 
use delayed feedback. 


22 For retention of procedural 
or conceptual knowledge, 
use immediate feedback. 


23 To promote transfer of 
learning, consider using 
delayed feedback. 


Description and references 

Feedback can be delivered (or obtained) either immediately 
after some activity or delayed. Immediate feedback can help 
fix errors in real-time, producing greater immediate gains 
and more efficient learning, (Corbett & Anderson, 2001; 
Mason & Bruning, 2001), but delayed feedback has been 
associated with better transfer of learning (e.g., Schroth, 
1992). 

When a student is learning a difficult new task (where difficult 
is relative to the learner’s capabilities), it is better to use 
immediate feedback, at least initially (Clariana, 1990). This 
provides a helpful safety net so the learner does not get 
bogged down and/or frustrated (Knoblauch & Brannon, 
1981). 

When a student is learning a relatively simple task (again, 
relative to capabilities), it is better to delay feedback to 
prevent feelings of feedback intrusion and possibly 
annoyance (Clariana, 1990; Como & Snow, 1986). 

In general, there is wide support for use of immediate feedback 
to promote learning and performance on verbal, procedural, 
and even tasks requiring motor skills (Anderson et al., 2001; 
Azevedo & Bernard, 1995; Corbett & Anderson, 1989, 

2001; Dihoff et al., 2003; Phye & Andre, 1989). 

According to some researchers (e.g., Kulhavy et al., 1985; 
Schroth, 1992), delayed may be better than immediate 
feedback for transfer task performance, although initial 
learning time may be depressed. This needs more research. 
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Table 5 

Formative Feedback Guidelines in Relation to Learner Characteristics 



Prescription 

Description and references 

24 

For high-achieving learners, 
consider using delayed 
feedback. 

Similar to the Clariana (1990) findings cited in Table 4, high- 
achieving students may construe a moderate or difficult task as 
relatively easy and hence benefit by delayed feedback (see 
also Gaynor, 1981; Roper, 1977). 

25 

For low-achieving learners, 
use immediate feedback. 

The argument for low-achieving students is similar to the one 
above, only these students need the support of immediate 
feedback in learning new tasks they may find difficult (see 
Gaynor, 1981; Mason & Bruning, 2001; Roper, 1977). 

26 

For low-achieving learners, 
use directive (or corrective) 
feedback. 

Novices, or struggling students, need support and explicit 
guidance during the learning process (Knoblauch & Brannon, 
1981; Moreno, 2004), thus hints may not be as helpful as more 
explicit, directive feedback. 

27 

For high-achieving learners, 
use facilitative feedback. 

Similar to the above, high-achieving students or more motivated 
ones benefit from feedback that challenges them, such as hints, 
cues, and prompts (Vygotsky, 1987). 

28 

For low-achieving learners, 
use scaffolding. 

Provide early support and structure for low-achieving students 
(or those with low self-efficacy) to improve learning and 
performance (e.g., Collins et al., 1989; Graesser, McNamara, 

& VanLehn, 2005). 

29 

For high-achieving learners, 
verification feedback may be 
sufficient. 

Flanna (1976) presented findings that suggest that high-achieving 
students learn more efficiently if permitted to proceed at their 
own pace. Verification feedback provides the level of 
information most helpful in this endeavor. 

30 

For low-achieving learners, 
use correct response and 
some kind of elaboration 
feedback. 

Using the same rationale as with supplying scaffolding to low- 
achieving students, the prescription here is to ensure low- 
achieving students receive a concrete, directive form of 
feedback support (e.g., Clariana, 1990; Flanna, 1976). 

31 

For learners with low 
learning orientation (or high 
performance orientation), 
give specific feedback. 

As described in the study by Davis et al., (2005), if students are 
oriented more toward performance (trying to please others) 
and less toward learning (trying to achieve an academic goal), 
provide feedback that is specific and goal directed. Also, keep 
the learner’s eye on the learning goal (Floska, 1993). 
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5.2 Future Research 

One reason why studies examining fonnative feedback effects are so inconsistent may be 
due to individual differences among motivational prerequisites (e.g., intrinsic motivation, beliefs, 
need for academic achievement, academic self-efficacy, and metacognitive skills). In fact, 
Vygotsky (1987) noted that the study of psychology had been damaged by the separation of the 
intellectual from the motivational and emotional (or affective) aspects of thinking. Crafting and 
delivering fonnative feedback may help bridge these aspects of thinking and enhance learning. 
This seems to be supported by a growing number of researchers (e.g., Goleman, 1995; Mayer & 
Salovey, 1993, 1997; Picard et ah, 2004) who have argued that emotional upsets can interfere 
with mental activities (e.g., anxious, angry, or depressed students do not learn). Thus one 
intriguing area of future research is to systematically examine the relationship(s) between 
affective components in feedback and outcome performance. And while there have been inroads 
in the area, according to Picard et ah, extending cognitive theory to explain and exploit the role 
of affect in learning is still in its infancy. 

Schwartz and White (2000), suggested, in general, a multidimensional view of feedback 
is needed, where situational and individual characteristics of the instructional context and learner 
are considered, along with the nature and quality of a feedback message. Narciss and Huth 
(2004) noted, and I strongly agree, that function, content, and mode of feedback presentation are 
important facets and should be considered separately, as well as interactively, with learner 
characteristics and instructional variables. Cognitive task and error analyses may be used to 
match formative feedback components to (a) learning objectives, (b) skills needed for the 
mastery of the task, and to (c) typical errors or incorrect strategies. However, such expensive 
analyses and methods may not, in fact, be necessary to promote learning, (e.g., see Section 3.5 
on the Sleeman et ah, 1989, findings). 

In line with the question concerning the added value of employing error analyses and 
more diagnostic types of formative feedback, controlled evaluations are needed to systematically 
test the effects of feedback conditions (as listed in Table 1) on learning, combined with a cost- 
benefit analysis. Some obvious costs include development time for specifying the feedback types 
and time for the student to read the feedback. Benefits relate to improvements in learning 
outcome and efficiency, as well as possibly affective variables. Infonnation about the learner 
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would be collected to examine possible interactions. The hypotheses are that (a) more complex 
formative feedback types (e.g., involving extensive and expensive error analyses) do not yield 
proportionately greater learning gains, and (b) feedback can be made more effective if it can 
adapt to the particular needs of learners—cognitive and noncognitive characteristics—as well as 
to different types of knowledge and skills. The general question is: What level of feedback 
complexity yields the most bang for the buck? 

Table 5 provides guidelines for linking a few learner characteristics to different feedback 
types. Future research may examine (a) additional learner characteristics, as well as (b) links 
between different types of knowledge and feedback types. For instance, feedback to support fact 
learning (declarative knowledge) could reiterate definitions or provide the learner with a handy 
mnemonic technique; feedback to support conceptual knowledge could provide examples, 
counter-examples, and big pictures; and feedback to improve procedural knowledge could 
involve demonstrations, solution paths (complete or partial), and so forth. Ultimately, 
information about the learner, combined with information about desired outcomes, may inform 
the development of adaptive formative feedback. Various feedback types could be generated and 
incorporated into a program (or generated on the fly based on formative feedback models) and 
then accessed and delivered according to the characteristics of the learner in conjunction with the 
nature of the task and instructional goals. 

In closing, the goal of this review was to summarize research findings relating to 
formative feedback to serve as the foundation for a variety of future educational products and 
services. As evidenced throughout, there is no best type of fonnative feedback for all learners 
and learning outcomes. However, fonnative feedback has been shown, in numerous studies, to 
improve students’ learning and enhance teachers’ teaching to the extent that the learners are 
receptive and the feedback is on target (valid), objective, focused, and clear. 
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Notes 


'An interesting observation is that many field studies demonstrate the value of immediate 
feedback (see Kulik & Kulik, 1988), while many laboratory studies show positive effects of 
delayed feedback (see Schmidt & Bjork, 1992; Schmidt et ah, 1989). 
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